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We consider renewal stochastic processes generated by non-independent events from the perspec- 
tive that their basic distribution and associated generating functions obey the statistical-mechanical 
structure of systems with interacting degrees of freedom. Based on this fact we look briefly into the 
less known case of processes that display phase transitions along time. When the density distribu- 
tion ip„(t) for the occurrence of the n-th event at time t is considered to be a partition function, 
of a 'microcanonical' type for n 'degrees of freedom' at fixed 'energy' t, one obtains a set of four 
partition functions of which that for the generating function variable z and Laplace transform vari- 
able e, conjugate to n and t, respectively, plays a central role. These partition functions relate to 
each other in the customary way and in accordance to the precepts of large deviations theory, while 
the entropy, or Massieu potential, derived from ijj n {t) satisfies an Euler relation. We illustrate this 
scheme first for an ordinary renewal process of events generated by a simple exponential waiting 
time distribution ip{t). Then we examine a process modelled after the so-called Hamiltonian Mean 
Field (HMF) model that is representative of agents that perform a repeated task with an associated 
outcome, such as an opinion poll. When a sequence of (many) events takes place in a sufficiently 
short time the process exhibits clustering of the outcome, but for larger times the process resembles 
that of independent events. The two regimes are separated by a sharp transition, technically of the 
second order. Finally we point out the existence of a similar scheme for random walk processes. 

PACS numbers: 02.50.-r, 05.20.-y, 05.70.Fh 
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I. INTRODUCTION 

A large class of stochastic processes are renewal pro- 
cesses [l| 0. This class of sequences are generally used 
to model independent identically distributed (iid) occur- 
rences. The renewal processes are concerned with the 
times of substitution of components that are replaced as 
soon as they break down. Here we recall 0] Q that 
this basic type of stochastic process possesses all the el- 
ements of a statistical-mechanical system and therefore 
can be couched into this language and benefit from well- 
established methods and applications developed for the 
study of systems with many degrees of freedom. The 
common iid process maps into the non-interacting case, 
but the most important potential application of this 
equivalence is to the generalization of renewal processes 
to correlated events, where the large body of knowledge 
accumulated in the study of (short or long range) inter- 
acting particle or spin systems can find interesting guide- 
lines or clear-cut analogies for renewal processes. One 
particular property that we present here is the occur- 
rence of phase transitions along time evolution. 

The layout of the article is as follows. We start in 
Section 2 with a concise description of a renewal pro- 
cess that involves the transformation into Laplace space 
of the relevant probability density functions and the use 
of generating functions [5]. In Section 3 we make ex- 
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plicit the statistical-mechanical ensemble structure, with 
only two pairs of conjugate variables, of the renewal pro- 
cess and illustrate the form that the partition functions 
take for the simple case of an exponentially-decaying 
waiting-time distribution density. In Section 4 we ap- 
ply the saddle-point approximation in the evaluation of 
the partition functions and show that the required Leg- 
endre transform structure, the associated equations of 
state, and the Euler relation [6| are present in the for- 
malism for the renewal process. In Section 5 we consider 
a specific example of a renewal process with correlated 
events that exhibits a phase transition when the time 
variable increases. The renewal process is representative, 
for instance, of an opinion poll, and is constructed to 
be equivalent to the statistical-mechanical Hamiltonian 
Mean Field (HMF) model of interacting particles 043 ■ 
Finally in Section 6 we summarize and discuss our re- 
sults. 



II. BASICS OF RENEWAL PROCESSES 

Technically, an ordinary renewal process is a sequence 
of partial sums of iid positive random variables. This 
process may be thought of as a sequence of points in time 
when the lifetimes of some objects of the same type ends 
and they are replaced by new ones. The renewal process 
counts the number of renewals in the interval [0, t), hence 
such a renewal counting process is a random piecewise 
constant function. A convenient analytical procedure to 
determine the properties of this kind of process is that 
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of Montroll [5]. It is resumed as follows: Let ip{t) be 
the (normalized) waiting time distribution density for a 
single event and i/j n (t) the distribution density for the 
occurrence of the n-th event at time t. For iid events 
these densities are linked via 



xjj n (t) = / dt' j>(t - t') i> n -i(t'), n > 1, (1) 



or in Laplace space by 
where 



^(e) = J dt exp(-ei) ip(t) 
o 

and 

oo 

$n(e) = J dt exp(-et) i/} n (t). 
o 



(2) 



(3) 



(4) 



We shall consider throughout this paper time variables 
to be dimensionless. A generating function for the ip n (t) 
is defined via the z-transform 



i;(t;z) = J2Mt) z n , 



(5) 



n=l 



so that 



°° oo 

$(e; z) = dt cxp(-ei) i/j(t; z) = ^ ^ n (e) z™. (6) 



Use of Eq. ([5]) above turns ^(e; z) into a geometric series 
that when convergent becomes 



V>(e;z) = V»(e) z / 1- $(e) z 



(7) 



The functions ipn(t) and ^>(£; z) are recovered from ip n (e) 
and ^(e; z), respectively, via inverse Laplace and inverse z 
transforms. The average time between events, or period, 
T is given by the first moment of ip(t), 



T= dtt tp(t) = — rlnV'(e) 
' de 



< oo, (8) 



e=0 



whereas the average number of events (n(t)} of a renewal 
sequence when the last event occurs at time t [IFJ] is 



(9) 



Therefore, if the most common calculation aim is to de- 
termine ipn(t) or (n(t)} for any given waiting time tp(t) 
distribution, use of ip{e) in Eqs. © and (J7J followed by 
inverse transformation is an expedient method. 

Here we recapture 0] 01 a precise interpretation of 
the above expressions while calling attention that it is 
not restricted to iid processes. This is that the functions 

ipn(t), ipn(t), an d VK 6 ! z ) can be seen to be par- 

tition functions associated to an equilibrium statistical- 
mechanical system of n degrees of freedom arranged in 
configurations with energy measured by a time t. Below 
we detail that in the large t and n limits these func- 
tions can be evaluated via the saddle-point approxima- 
tion and that this central statistical-mechanical property 
leads to equations of state and entropies or free energies 
related via Legendre transforms, where the variables e 
and n = lnz appear to be conjugate to the variables t 
and n, respectively. 



III. STATISTICAL ENSEMBLES FOR 
RENEWAL PROCESSES 

We observe that Eq. ©, V>(e;z) = J2™=i^n(t) z n , 
has the form of the expression for the grand canonical 
partition function of a thermal system if we were to con- 
sider that the number of events n represents the number 
of particles or degrees of freedom, e the inverse tempera- 
ture, z the activity, and therefore V> n (e) plays the role of 
the canonical partition function. Having considered that 
^Vs(e)i the Laplace transform of tp n (t) m Eq. (@|, plays 
this role, the formal analogy can be extended by identifi- 
cation of ip n {£) as the microcanonical partition function 
where t is the energy. Further, the generating function 
ip(t; z) would then be seen as the partition function cor- 
responding to an ensemble of fixed energy t and activity 
z. (For iid random variables ip n ( e ) is given by Eq. @ 
and the corresponding thermal system is made of iden- 
tical non-interacting degrees of freedom with the 
canonical partition function per degree of freedom). 

The scope of this analogy can be further assessed b 
defining the following entropies or Massieu potentials 



hiip(e;z), S ein = hitp n (e), 



St,n = hxip(t;z), S t , n = ln^ n (*)) 



(10) 



where we have introduced the 'chemical potential' \x = 
lnz. (These quantities may be negative since the argu- 
ments of the logarithms may be less than unity. Notice 
that these arguments are probability densities or their 
Laplace and/or z-transforms, while in ordinary statisti- 
cal mechanics the arguments are configuration numbers 
or their transforms). If for large n a thermodynamic 
limit or a large deviations property arises, then these 
potential functions would be related via Legendre trans- 
forms involving the pairs of conjugate variables (n, fi) 
and (t, e) and mediated via the corresponding equations 
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of state. An Euler relation of the type 
St, n =te — nfj, 



(11) 



would hold, and attention should be paid in the evalu- 
ation of = \nip(e;z) as a cursory inspection of re- 
peated Legendre transforms would imply S €lfi = St, n ~ 
te + n\i = 0. Below we show that Eq. (fTTj) holds with a 
nonzero S Etti . 

To help us examine the validity of this formal structure 
in the following section we determine the above partition 
functions for the particular iid random variable case of 
an exponential waiting-time density ip(t) = &exp(— bt). 
One obtains 



and 



?/>(e; z) = bz (b + e — bz) 1 , 



i }n {e)=b n {b + e)- n , 



ip(t; z) = 6zexp(— bt + btz) 



i t = i-f- exp(-W), 

71! 



(12) 



(13) 



(14) 



(15) 



where we recognize in the last equation the distribution 
density of a Poisson process. 



IV. ANALOGY WITH STATISTICAL 
MECHANICS 

The asymptotic solution of VVi(e) for n » 1 can be 
found by use of the steepest-descent approximation of the 
inverse z-transform of ip(e; z), 

^n(e) = <j> dz exp [(n - 1) (- lnz+ 

(n-lj^ln^z))], (16) 

One obtains 

ln$n(e) — ~{n - l)mz + lnip(e;z ), (17) 

where zq can be eliminated in favor of n via the steepest- 
descent condition 



1 



z— lni/)(e; z) 
dz 



z=z 



It is then possible to write 

V> n (e) = expS e 



(18) 



(19) 



where S e . n = ln^^e) is the Legendre transform S e ^ n = 
— (n—l)fj,Q+S 6j fj, of S €tl j, — hiip(e; zq). For the exponen- 
tial waiting-time density ijj(t) — bexp(-bt) this transfor- 
mation leads to the equation of state 



and to the Massieu potential 
b n-i 



In 



(b + £)"-! (ra-2)"- 



(21) 



In the limit n — > oo Eq. ([13]) is recovered. 

Similarly, the asymptotic solution of ip n (t) for n >> 1 
is obtained with the use of the steepest-descent approxi- 
mation of the inverse Laplace transform of ip n (e) , 



1 

2iri 



c+ioo 



de exp 



n (eT + ln${e)) , (22) 



where r = t/n. One obtains 

rT x In ^)„(nr) ~ eo r + hi^(eo), ti >> 1, (23) 

where eo can be eliminated in favor of r via the steepest- 
descent condition 



r=--ln^(e) 



(24) 



In taking the limit n — > oo also t — > oo but r is kept 
finite. We can therefore write 



tp n (t) = expS t , n , 



(25) 



where St, n — hif/Vi^) is the Legendre transform St, n — 
—teo + S €0tn of S €0tTl = In V'ra(eo). For exponential waiting 
times = 6exp(— 6t) this transformation leads to the 
equation of state 

t = n(b + e )-\ (26) 
and to the Massieu potential 

S ttn = hx (btn~ 1 ) n exp(n) exp(-bi) . (27) 

Therefore Eq. (| 1 5[) is recovered in the limit n — > oo (when 
we notice that the Stirling approximation of the factorial 
is part of Eq. ([27]) ). 

Lastly, following an analogous procedure the asymp- 
totic form for ip(t\ z) for n » 1 is given by 

ip{t',z)=expSt, li , (28) 

where the Massieu potential St iM for tp(t) — bexp(-bt), 

S t ,n = ki[btzexp(l + bt + btz)], (29) 

is obtained as the Legendre transform St }fi = tea + S eo ^ z 

Of Se, 



, ea ^ — ln^(eo; z) with 



< = (b + Eq — bz) 1 



(30) 



1 = 1 + bz (b + e - bz y 



(20) 



Eq. ([2"9"| is consistent with Eq. ([14]) in the limit n — >• oo. 

To make explicit the observance of the Euler relation 
Eq. pip we note that the inverse Legendre transform 
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that yields S ejfi from S e>no , 5 £ , M = (no - + <5 e ,„ D , 
requires 



H = 



so that S e 



d(n - 1) 



nln V'( e ) 



= -ln^(e), (31) 



n In ^(e) = — n /J, and S e ^ = (n — l)^ + 



Se,n = —fJ" This leads to 

St ,n = *oe - («o - 1)M + 



nop. 



(32) 



We note that the existence of the Euler relation for a 
system with only two pairs of conjugate variables does 
not imply the vanishing of the thermodynamic potential, 
Se,fj,, associated to two consecutive Legendre transforms 
of the basic potential, St, n , a homogeneous function of 
order one in both variables t and n 6]. Notably, the 
partition function tp(t] z) associated to the variables e 
and fj, remains a fundamental and most useful quantity 
for the description of the renewal process. 



V. AN EXAMPLE OF A RENEWAL PROCESS 
WITH CORRELATED EVENTS 

As an illustration of how developments in the statis- 
tical mechanics of interacting particle or spin systems 
may have meaningful translations to renewal processes 
we present here features of a renewal process with cor- 
related events. We take inspiration from the so-called 
Hamiltonian Mean Field (HMF) model QO to point 
out the occurrence of phase transitions along time evolu- 
tion. 

Consider a sequence of events, each of which, besides 
taking place at a given time t, assigns values to two 
scalar quantities r and 9, the first within the time in- 
terval < r < t taken by the event, and the second 
within a fixed finite interval, say < 9 < 2n. For in- 
stance, the process may represent an agent (or agents) 
that performs a repeated task with outcome (t, 9) that 
is not independent of those for all the previous events. 
A string of such n events is described by the sequence of 
triplets [(ti; ti, 6>i), (t n ; r„, 6 n )\. The two collections 
of values (t%, T2,..., t„) and (6*i, 02,.-, @n) are used to 
construct two additional time variables, T n , the "idle" 
time, and W n , the "active" time, respectively, that to- 
gether comprise the total time taken by the sequence of 
n events, i.e. 



t n =T n + W n . 
The idle time T„ is simply given by 



T„ 



i=l 



(33) 



(34) 



whereas the active time W n measures the dispersion of 
the values (9x, &2,---, 9 n ) over the unit circle, being large 
when these are spread out over (0, 2tt) and small when 



they concentrate around a given 9. Although there are 
many options to define W n , for definiteness we chose it 
to be determined by 



1 



Wn = ^Y, [l-cos(|0«-fy|)]<t, 



(35) 



where all pairs (9i, 9j) are equally considered. (We re- 
call that time variables are considered dimensionless) . 
Clearly, the condition Eq. (|33|) imposes a restriction in 
the possible values of the sequences (ti, T2,..., t„) and 
(9i, #2v) 9n)- As a more specific illustration of this 
kind of process let us suppose there is an opinion poll 
organization that sends an agent (or group of pollsters) 
to take a survey that consists of a succession of n com- 
pleted questionnaires obtained in the time interval (0,i). 
Each respondent has a tag 9 that quantifies a charac- 
teristic of the population surveyed, such as age, race, 
home environment, etc., and therefore W n reflects the 
degree of coverage bias, in, for instance, the considera- 
tion of young voters, minorities, or rural areas. The time 
t associated to each respondent measures wasted time 
in collecting opinions, since some people do not answer 
calls, or refuse to answer the poll, or do not give can- 
did answers, and consequently T n represents the extent 
of nonresponse bias. 

The probability density of occurrence of the n-th event 
at time t with outcome (0, t), ip n (t;T,0), is evaluated 
in terms of the statistics of occurrence of the previ- 
ous n — 1 events. This is best prescribed in terms of 
the Laplace transform of tp n (t;T,9) with respect to t, 
ip n (e;T,9). Specifically, the renewal process is given by 



Tp n {e; t, 9) ee exp [-e (St n (r, 0)) n _ x ] 



(36) 



where the average (— is performed over all possible 
values of (n,0i), (t„_i,0„_i), and 



St n (r, 9) = T n + W n - T n _! - W n -i. 



(37) 



The analogy with the HMF model becomes evident 
when it is seen that Eq. (|36[) corresponds to Widom's 
particle insertion formula when applied to the thermal 
system (for vanishing chemical potential fi) [12] [3 ■ The 
roles of the number of particles, their positions (in the 
unit circle), inverse temperature, energy, kinetic energy, 
and potential energy of the HMF model, are given, re- 
spectively, by n, 9i, e, t n , T n , and W n . (With no loss 
of generality a coupling constant in the potential energy 
term of the ferromagnetic HMF model has been set equal 
to unity). In our notation, the Helmholtz free energy of 
the HMF model in the limit n — > oo, obtained via the 
saddle-point approximation is 

A 1 / 6 \ 6 / £X^ 

-ei/j„(e) = --In J — — +max ( — — + In 2irI Q (ex) 

(38) 

where the auxiliary variable x satisfies 

h(ex) 

X = 6 T I \ 

I [ex) 



(39) 
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Ii{y} is the modified Bessel function of order 
vn 



and where 

i. As known [7|-|9j], the HMF model exhibits two equi- 
librium phases, indicated by the possible solutions of Eq. 
([55)) . When e < e c — 2 the variable x, identified as the 
model's magnetization M, vanishes, but x is nonzero for 
e > e c , increasing gradually as e increases and reaching 
unity as e — > oo. These properties imply that the mean 
active time (W n ), the average of W n over all sequences 
(tx,9i), (t„, 6>„), is given by (W n ) = (1 - a; 2 )n/2, and 
that the relationship between the time t and the ampli- 
tude e (the caloric equation for the HMF model) reads 



its transition at e c is reflected by the ^-dependence of 
^ n (e)T,9) when n 3> 1. For small e the distribution is 
uniform in 9, but when e > e c it becomes peaked around 
a given (although arbitrary) value of 9 = <f». As we see 
below this feature is preserved in the distribution for the 
original variable t, i.e. there is a critical time t c above 
which ip n (t;T,9) is uniform in 9 and below which it is 
peaked around a given 9 — cj>. The r-dependence of 
ip n (e)T,0) has an exponential form (Gaussian if writ- 
ten for the 'momentum' p= ±\/2t ) for all e. When 
n -t oo Eq. (J3S) leads to [H 



t = - + (w n ), 



(40) 



4> n (e; r, 9) = J ^- cxp (-er) - / exp(ex • 6>), (41) 



Thus, (W n ) displays a fixed maximum value where x = (xcos(/>, xsin<j)) and ^ = (cos 9 : sin ^ 





i.o 
t/n 



FIG. 1. Dependence of t/n, (,n>l, on the Laplace variable 
e for the renewal process model of correlated events designed 
to be analogous to the HMF model. The figure is equivalent 
to the caloric equation of the HMF model and shows the two- 
phase behavior described in the text. 



(W n ) = n/2 for e < 
approaches zero as e - 



= 2, whereas it decreases and 
xd. The two-phase behavior and 

I 



FIG. 2. Dependence of the active time per unit event (W n )/n 
for n^> 1 on t/n for the renewal process model of correlated 
events designed to be analogous to the HMF model. The 
figure shows the two-phase behavior described in the text. 



The corresponding expression for i/j n (e;T,9), the inverse 
Laplace transform of Eq. (|4"Tj) obtained via the saddle- 
point approximation, is 



tp(t;r,e) ~ Cexp (x 2 - 1/2) 



1/2 



1/2 



Io 



l/2)x 



(42) 



r 



where C is a normalization constant. Following Refs. [8[ 
@ we evaluated the dependence of (W n ) /n on e after 
solving numerically Eq. (|55|) . Subsequently we used this 
in Eq. (|40p to obtain the dependence of t/n on e (shown 
in Fig. 1, where the two-phase feature is evident). This 
allowed us to determine the time dependence of the mean 



active time per event, (W n ) /n, shown in Fig. 2, where 
it is observed that this quantity increases monotonically 
with t until it saturates at a value of 1/2 at t c and re- 
mains constant thereafter. Since (W n ) /n measures the 
average spread of the tags 9i, i — l,...,n, we conclude 
that for short times t < t c this spread falls below its 
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-2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 







FIG. 3. Dependence of t[> n (t;T,9), n > 1, on the 8 when 
t < t c . When t > t c the function ip n (t;T,8) is ^-independent. 



maximum whereas for larger times t > t c the maximum 
spread is always assured. In terms of the opinion poll 
sets of n samples taken inside time intervals t < t c suffer 
from coverage bias but are as free of it as it is possible 
when the set of samples are collected within time inter- 
vals t > t c . This feature is corroborated in Fig. 3 where 
we show the ^-dependence of tp n (t\ t,6), t < t c , as given 
by Ec. (l42l) . When t > t c the density is flat, independent 
of 9. Interestingly, sequences of n events that take place 
within time intervals t < t c are correlated while those for 
t > t c are not. When (W n ) jn < 1/2 there is on average 
no sufficient time for the pollster to move to other loca- 
tions or to switch to different population groups, there is 
a repetition, or ordering in the set of samples. This gen- 
erates a coverage bias. Similar arguments can be elabo- 
rated in terms of the average idle time (T„) /n = l/(2e) 
that in the example of an opinion poll is reflected by the 
presence of nonresponse bias. A measure of the correla- 
tions induced for t < t c is given by the time derivative of 
(W n ) /n (one of several response functions) as shown in 
Fig. 4. For t > t c the HMF model behaves effectively as 
an ideal gas, and, as we can see from Figs. 2 to 4, the 
renewal process conforms to that of independent events 
for this regime. 

Thus, by construction our opinion poll renewal process 
acquires all the properties of the HMF model, mainly its 
second order phase transition that separates two different 
regimes. That is, for small t/n strings of n events cluster 
around a given value of the tag 9 symptomatic of an 
inefficient poll, but for larger t/n the events display a 
uniform dispersal of 9 suggesting the proper working of 
the sampling process. The clustering of the tag 9 when 
e > e c is expressed by the Laplace transform variable e 
as it measures the width of ip n (e;T,9). There are other 
known interesting properties of the HMF model such as 
the occurrence of long lived, or quasistationary, states for 
temperatures below the transition temperature, when the 
system displays features of the high-temperature phase 
uniform in 9 Q These states would manifest also in 
the renewal process as sequences of active times (W n )/n 
larger than those shown in Fig. 2 for some range of values 



CO 




0.0 0.2 0.4 0.6 0.8 1.0 1.2 

t/n 

FIG. 4. Dependence of d((W n )/n)/d(t/n), t,n > 1, on t/n. 



t < t c close to t c 



VI. SUMMARY AND DISCUSSION 

We have made use of the statistical-mechanical inter- 
pretation of the basic elements that constitute the the- 
ory of renewal processes. Our purpose for recapturing 
this analogy is to facilitate the application of useful tech- 
niques and approximations built up and tested through a 
large amount of studies of thermal systems. Potentially 
these methodologies can have important effects in the 
study of complex systems that originate outside ordinary 
statistical-mechanical physical systems, in a variety of 
fields, in ecology, economy, sociology, etc. where stochas- 
tic processes such as that for the renewing of events often 
arise. The known parallels between renewal processes 
and statistical mechanics are an indication of the gen- 
eral, Laplace and Leg endre transform structure of large 
deviations theory [111 ]. The saddle-point approximation 
is central to this theory where a probability P n obeys 
the form P n ~ exp(— ns) for n 3> 1 with s a positive 
quantity independent of n named the rate function [TTI ] . 
Clearly, the Massieu potentials S e _ n and St, n in Eqs. (Til))) 
and (|25[) . respectively, when written as S e _ n — —ns(e) 
and St >n = —ns(r) comply with this property. Thus we 
could describe renewal processes, familiar in the proba- 
bility theory domain, in this alternative language. Nev- 
ertheless, because of our stated purposes we have used 
a statistical-mechanical language. We have pointed out 
the equivalence with large deviation theory when appro- 
priate. As a difference from the present statistical study 
of single sequences of correlated events, sets of renewal 
sequences dependent on each other (when uncoupled the 
sequences are made of uncorrelated events) have been an- 
alyzed with the use of multivariate distributions. For a 
recent application see Ref. 14 1. 
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To exemplify the use of the parallelism between re- 
newal processes and statistical mechanics we devised a 
model renewal process with correlated events that dis- 
plays a phase transition as time progresses. When n 
events take place within a relatively short time interval 
their correlation is evident, whereas for longer time in- 
tervals their statistical properties are identical to those 
of independent events, and there is a sharp transition 
between the two regimes. The renewal process ensemble 
structure facilitated the description of this model that 
we chose to portray, amongst several possible options, 
in terms of a polling process, and with characteristics 
taken straightforwardly from a well-known particle or 
spin statistical-mechanical model, the HMF model Q- 
There are examples of phase transitions occurring along 
time evolution in deterministic (as opposed to stochastic) 
systems. See Ref. |l5[ and references therein. 

We close by mentioning that correlated random walk 
processes on regular lattices, as described with the help 



of the Fourier transform and generating functions [5| , and 
for both discrete and continuous time distributions, can 
be couched into a partition function language just as we 
have shown here for correlated renewal processes. Due 
to the sign of the integers used to locate the walker in 
lattice space the ensuing statistical-mechanical formalism 
differs also from the canonical type, basically by using 
the velocity instead of the kinetic energy as the primary 
variable that describes interacting particles. 
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